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Ramp-Up to Readiness™ (Ramp-Up) is a program for middle and high school students 
that aims to provide greater depth and breadth of support for college readiness than 
do current supports. This study examined whether students’ ACT Engage goal setting 
and commitment to college scale scores as well as the likelihood of key enrollment 
actions such as completion of a financial aid application and submission of at least 
one college application increased more in public high schools implementing Ramp-Up 
for one year than in similar schools that did not implement the program. The results of 
this cluster randomized controlled trial found no statistically significant differences on 
the outcomes between students in the two groups of schools. However, because the 
sample of students was smaller than anticipated, if Ramp-Up had some modest-sized 
impacts, the study would not have been able to detect them. 


This brief summarizes the findings of Lindsay, J., Davis, E., Stephan, J., & Proger, A. (2017). Impacts 
of Ramp-Up to Readiness™ after one year of implementation (REL 2017-241). Washington, DC: U.S. 
Department of Education, Institute of Education Sciences, National Center for Education Evalua- 
tion and Regional Assistance, Regional Educational Laboratory Midwest. That report is available at 
http://ies.ed.gov/ncee/edlabs/projects/project.asp?projectID=1461. 
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Why this study? 


College education is fundamental to students’ upward mobility, states’ economic growth, and the coun- 
try’s economic competitiveness (Bureau of Labor Statistics, 2013; Hanushek & Kimko, 2000; Hanushek 
& Woessmann, 2012). Ninety-three percent of graduating seniors plan to enroll in college (Ross et al., 
2012), yet 79 percent of students in the United States do so by age 20 (Center for Public Education, 2014). 
Of the students who enroll, only 65 percent attain a postsecondary certificate or degree (Snyder & Dillow, 
2015). Faced with this gap between high school students’ college aspirations and the actual percentage 
of students who attain a postsecondary degree, policymakers are expecting K-12 school systems to better 
prepare students to enroll and succeed in college. Several strategies have been adopted to respond to these 
expectations, including requiring students to complete postsecondary plans, adopting academic standards 
that are better aligned with college expectations, offering students more opportunities for dual enrollment, 
and improving the college advising process. 


This report describes the impact of a program—Ramp-Up to Readiness™ (Ramp-Up)—that takes a 
schoolwide approach to improving the college readiness of high school students. Members of the Midwest 
College and Career Success Research Alliance expressed an interest in learning how the program attempts 
to improve students’ college readiness, how it differs from typical college-readiness supports in high schools, 
how it is implemented, whether schools meet the consortium’s expectations for implementation, how school 
staff perceive the program, and whether the program had an immediate impact on student outcomes. Since 
2012 Regional Educational Laboratory Midwest has worked with alliance members to answer these ques- 
tions. This report presents information on the program’s immediate impacts on students in a sample of 49 
schools and on the quality of program implementation after a single year in a sample of 25 schools. 


Ramp-Up is a research-based program developed by the University of Minnesota’s College Readiness 
Consortium. It aims to increase middle and high school students’ likelihood of college enrollment and 
completion by enhancing five dimensions of college readiness: academic, admissions, career, financial, and 
personal-social (box 1). Ramp-Up engages all students in a school by training the majority of school faculty 
to facilitate 28-30 minute grade-specific advisory sessions (approximately one per week) and five class 
period-long workshops throughout the year. The Ramp-Up advisory sessions and workshops all connect 
to two student tools: a postsecondary plan and a readiness rubric to measure progress on the plan. In their 
postsecondary plan, students describe their college and career aspirations, their planned coursework each 
year, and their extracurricular activities. Students use the readiness rubric to measure their progress on 
their plans. Students update the postsecondary plan once a year and update the readiness rubric three 
times a year. These tools are also shared with parents. 


Unlike many existing college-readiness interventions that focus on specific subgroups of students such as 
students from low-income households, students with high academic achievement, or students who self 
identify as college-bound, the Ramp-Up curriculum focuses on all students and is delivered during the 
regular school day. Even students who initially have no plans to attend college or knowledge about their 
postsecondary options take part in goal setting and receive instruction on how to prepare for college. 


Impacts of the program result, theoretically, from the cumulative growth in students across five dimensions 
of college readiness (see box 1 and figure 1). The consortium expects schools’ participation in Ramp-Up 
to improve immediate outcomes, such as the likelihood of completing a financial aid application (for high 
school seniors), taking the ACT or SAT, setting personal goals, and committing to college. In the long 
term the consortium expects that participation in Ramp-Up will increase students’ likelihood of enroll 
ing in a two- or four-year college, decrease the likelihood that students will need remedial coursework in 
college, and increase the likelihood that students persist in college. 


Box 1. Dimensions of college readiness 


The University of Minnesota’s College Readiness Consortium hypothesizes that Ramp-Up’s curriculum, tools, 

and professional development will increase college readiness by teaching skills and providing information, assis- 

tance, and encouragement in five interrelated dimensions of college readiness (College Readiness Consortium, 

2012): 

e Academic readiness: “The student has the knowledge and skills to do first-year, credit-bearing, college-level 
work” (p. 9). 

e Admissions readiness: “The student has completed all requirements for admission to the type of post- 
secondary education that is a match for their goals, interests and abilities” (p. 9). 

° Career readiness: “The student understands how education increasingly determines income and opportunity 
in the global knowledge economy, and will know which types of jobs in the future will need skilled workers, 
will pay enough to support a family and might be a good match for their interests and abilities” (p. 10). 

e Financial readiness: “Students will be able to cover the cost for one term of study [that is, a degree program] 
at a postsecondary institution through savings, loans, work-study, and financial aid” (p. 10). 

e Personal and social readiness: “The student knows how to set educational goals, make progress toward 
those goals, and create relationships with peers and adults that support the achievement of those goals” 
(p. 11). 


Source: College Readiness Consortium, 2012. 


Despite Ramp-Up’s potential to increase students’ college readiness and likelihood of college enrollment 
and completion, little information is available about how well schools are able to implement the program, 
and no information exists about the program’s impacts. In an earlier study of Ramp-Up, Regional Educa- 
tional Laboratory (REL) Midwest examined how the program differs from the college-readiness activities 
in other schools, whether staff in high schools were able to implement Ramp-Up at the level that the 
program’s developer believes is needed to achieve the proposed college-related outcomes, and school staff 
members’ opinions of the program’s strengths and weaknesses (referred to here as the Ramp-Up imple- 
mentation study; Lindsay, Davis, Stephan, Bonsu, & Narlock, 2016). The current study, referred to as the 
impact study, examined the immediate impacts of the program on two measures of students’ personal 
readiness for college and two key enrollment actions. This study represents a collaborative effort among 
the authors, members of the Midwest College and Career Success Alliance, the University of Minnesota’s 
College Readiness Consortium, colleagues at American Institutes for Research, and the staff in the 49 
Minnesota and Wisconsin high schools that participated in the study. This brief summarizes the full report 
of the impact study (Lindsay, Davis, Stephan, & Proger, 2017). 


This brief summarizes the findings for the study’s four research questions: 

e What is the impact of Ramp-Up on the likelihood of grade 12 students completing the Free Appli- 
cation for Federal Student Aid (FAFSA)? 

e What is the impact of Ramp-Up on grade 10, 11, and 12 students’ scores on the ACT Engage goal 
striving and commitment to college scales? 

e What is the impact of Ramp-Up on the likelihood of grade 12 students submitting at least one 
college application? (This question was exploratory because the program developer had no firm 
expectations that this outcome would be affected by the program within a single year.) 

¢ To what extent did schools implement the core components of Ramp-Up (structural supports, pro- 
fessional development, curriculum delivery, curriculum content, and postsecondary planning tools) 
as intended by the program developer? 


Figure 1. Ramp-Up to Readiness theory of action 
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Note: With high fidelity implementation and improvements in the dimensions of college readiness, the consortium expects im- 
mediate outcomes after one year of exposure to Ramp-Up, intermediate outcomes after two years of exposure to Ramp-Up, and 
long-term outcomes after three years exposure to Ramp-Up. 


a. The College Readiness Consortium at the University of Minnesota. 


Source: The University of Minnesota’s College Readiness Consortium. 


A randomized controlled trial was conducted to address the research questions after a single year of the 
intervention. Half of the participating schools were randomly assigned to begin implementing Ramp-Up 
during the 2014/15 school year and the other half continued their normal college readiness programming 
until 2015/16. See box 2 for a summary of the data and methods used in the study and appendix B in the 
full report for more detail (Lindsay et al., 2017). 


What the study found 


After one year of implementation Ramp-Up did not have an impact on students’ ACT Engage goal striving 
scale score, ACT Engage commitment to college scale score, likelihood of completing the FAFSA, or likeli- 
hood of submitting at least one college application. On average, all but one Ramp-Up school implemented 
the program at a level that the consortium considered adequate, but implementation of the components 
and subcomponents of Ramp-Up varied, with just 3 of 25 Ramp-Up schools adequately implementing all 
five of the key program components. 


After the year of implementation, students in Ramp-Up schools and students in comparison schools showed no 
differences in ACT Engage goal striving or commitment to college scale scores 


Following 8-9 months of exposure to the Ramp-Up curriculum, students in the Ramp-Up schools and students 
in comparison schools showed no statistically significant differences in ACT Engage goal striving or commit 
ment to college scale scores (figure 2). For both scales the average percentile score was the same for students in 
the two groups of schools (48th percentile; p = .854 for goal striving and p = .957 for commitment to college). 
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Box 2. Data and methods 


Sample of schools and students 

The consortium recruited 50 schools (48 in Minnesota and 2 in western Wisconsin) serving grades 10-12 to 
participate in this study. The study team randomly assigned 25 of the schools to implement Ramp-Up during 
the 2014/15 school year (the Ramp-Up group) and 25 schools to implement it during the 2015/16 school year 
(the comparison group). One comparison school was unable to provide the necessary data and was therefore 
dropped from the study. The two groups of schools were equivalent on baseline academic achievement indica- 
tors (such as state assessments and graduation rates) and demographics (Such as race/ethnicity of students 
and percentage of students eligible for the federal school lunch program). The two groups of schools did have 
different response rates on some measures, making it necessary to exercise caution when interpreting the 
findings. Dividing schools into groups allows for comparison of student outcomes between students in schools 
that implemented Ramp-Up and students in schools that did not. The final analytic sample consisted of 15,314 
students: 7,574 students in Ramp-Up schools and 7,740 students in comparison schools. 


Data collection 

The study team collected quantitative data (staff and student surveys and student records) and qualitative data 
(staff responses to open-ended survey items) at the beginning and end of the 2014/15 school year. To examine 
whether Ramp-Up had immediate impacts, the study team examined the rates of completion of the Free Applica- 
tion for Federal Student Aid (grade 12 students only), ACT Engage goal striving and commitment to college scale 
scores (grades 10-12), and the rates of submission of at least one college application (grade 12 students only). 


Data analysis 


Analysis of program impact. The magnitude of the impact of Ramp-Up was analyzed using hierarchical linear 
modeling, which is an accepted statistical approach that estimates program impacts while controlling for differ- 
ences within schools, differences between the groups of schools at baseline, and differences on other student 
and school characteristics. 


Analysis of implementation fidelity. To examine whether Ramp-Up schools implemented the program with fidel- 
ity (that is, as intended by the consortium), the study team coded staff members’ and students’ responses to 
surveys for the presence or absence of Ramp-Up activities in schools. The coded data were aggregated into a 
fidelity index representing the percentage of indicators present overall. The study team also calculated sepa- 
rate fidelity indexes for program components. The program developer independently established cutpoint index 
scores to distinguish schools that are implementing the program with excellent fidelity (an index at or above 
90 percent), adequate fidelity (an index of GO—89 percent), and inadequate fidelity (an index below 60 percent). 
The same cutpoints applied to program components. 


The differences in the likelihoods of completing the Free Application for Federal Student Aid and of submitting at 
least one college application between students in Ramp-Up schools and students in comparison schools were not 
statistically significant 


Likelihood of completing the Free Application for Federal Student Aid. After 8-9 months of exposure 
to the Ramp-Up curriculum, 58 percent of grade 12 students reported that they had completed the FAFSA 
(see figure 2). The percentage of grade 12 students in comparison schools was 51 percent. The difference 
was not statistically significant (p = 0.292). Had the study included a larger sample of students and schools, 
it is possible that this difference between the Ramp-Up and comparison school students would have been 
statistically significant.! 


Figure 2. The differences in outcomes after one year of implementation between students in 
Ramp-Up schools and students in comparison schools were not statistically significant, 2014/15 
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FAFSA is the Free Application for Federal Student Aid. 


Source: Authors’ calculations based on data from student surveys and ACT Engage surveys. 


Likelihood of submitting one or more college applications. Eighty-eight percent of grade 12 students in 
Ramp-Up schools reported submitting at least one college application, compared with 82 percent of grade 
12 students in comparison schools (see figure 2). The difference was not statistically significant once other 
student- and school-level factors were controlled for (p = 0.477).” 


All but one Ramp-Up school adequately implemented the program, but fidelity across program components varied; 
only 3 of 25 Ramp-Up schools adequately implemented all five program components 


Ramp-Up involves five core components, each of which includes a set of resources and activities provided 
by the consortium and enacted in schools: 

e Structural supports. Schools need to establish a Ramp-Up leadership team; appoint a Ramp-Up 
coordinator; enlist homeroom teachers to become Ramp-Up advisors; provide students with 
opportunities to enroll in advanced courses; provide support for professional development for 
administrators, teachers, and counselors; offer time for preparing and conducting advisory ses- 
sions and workshops; and adopt a technology platform for creating, storing, and sharing students’ 
postsecondary plans. Ramp-Up leadership teams consist of the principal, one counselor, one 
teacher, and any other suitable individuals. Their responsibilities include creating an annual plan 
and implementation calendar, guiding and monitoring implementation, attending training and 
workshops offered by the program developer, and providing professional development to all staff 
who deliver the program. 

e Professional development. Schools’ Ramp-Up leadership teams and coordinators need to participate 
in off-site professional development sessions led by the consortium, and school Ramp-Up leader- 
ship teams and coordinators must provide on-site professional development to staff who serve as 
Ramp-Up advisors. 


© Curriculum delivery. Ramp-Up advisors must have the necessary materials and information to 
understand the Ramp-Up curriculum, and they need to facilitate the 28 advisory sessions and 5 
workshops at the high school with the students in their homerooms. 

¢ Curriculum content. The content of the Ramp-Up advisory sessions and workshops must cover 
all five dimensions of college readiness. If Ramp-Up advisors follow the activities and workshops 
provided in their grade-specific advisor guides, students should perceive staff-student interactions 
related to each of the five dimensions. 

e Postsecondary planning tools. Ramp-Up advisors and students need to create a postsecondary plan, 
complete the consortium’s college-readiness rubric, review and update the plan and rubric given 
students’ experiences, and communicate students’ progress with their parents. 


On average, across components and schools, implementation was classified as adequate. A school’s 
overall implementation index is the percentage of indicators present across components and subcompo- 
nents of the program. The cutpoints established by the consortium were 90 percent or higher for excellent 
implementation fidelity, 60-89 percent for adequate fidelity, and less than 60 percent for inadequate fideli- 
ty. The average implementation index across Ramp-Up schools was 71 percent, suggesting that implemen- 
tation was adequate. However, school-specific indexes show that 24 of 25 schools (96 percent) implemented 
Ramp-Up with adequate fidelity, while the remaining school did not (figure 3). 


Only 3 of the 25 Ramp-Up schools had adequate implementation fidelity on all five components. While 
implementation was generally adequate, only three schools (12 percent) implemented all five components 
at adequate levels according to the program developer's standards (figure 4). 


Schools struggled especially with implementing the curriculum content and planning tools components. 
One of the main findings from the Ramp-Up implementation study (Lindsay et al., 2016) conducted in 10 


Figure 3. Ramp-Up schools had the most difficulty implementing the curriculum content and 
planning tools components with adequate fidelity, 2014/15 
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Source: Authors’ analyses of staff survey data, student survey data, and instructional logs from spring 2014. 


Figure 4. Only 3 of 25 schools (12 percent) adequately implemented all five Ramp-Up components, 
2014/15 
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Source: Authors’ analyses of school and developer implementation documents, staff survey data, student survey data, and in- 
structional logs from spring 2014. 


other schools was that schools struggled most with implementing the curriculum content and planning 
tools components with adequate fidelity. The Ramp-Up schools in the current study had similar difficulties 
with these two components (see figure 3). Sixty-three percent of the Ramp-Up schools were able to meet 
the consortium’s cutpoint for adequate implementation fidelity of the curriculum content component, and 
12 percent were able to do so for the planning tools component. Regarding the subcomponents of plan- 
ning tools, 60 percent of schools implemented the postsecondary plan with adequate fidelity, 52 percent 
of schools implemented the readiness rubric adequately, and none of the 25 schools implemented parent 
communication with adequate fidelity. 


Implications of the study findings 


The findings from this study have implications for the program developer, school leaders and staff currently 
implementing Ramp-Up, school and district leaders considering adopting Ramp-Up, and policymakers who 
want to improve students’ college readiness. These implications are summarized as follows: 


Implications for the creators of the program at the College Readiness Consortium 


The findings echo those from the Ramp-Up implementation study (Lindsay et al., 2016): schools were 
unable to implement all components with fidelity, especially the curriculum content and planning tools 
components. One possible implication of this finding is that the consortium needs to provide schools with 
more guidance on implementing particular components, including curriculum content and postsecondary 
planning tools. Schools implementing the program may need to ensure that advisories and workshops 
cover all five dimensions of college readiness, that students and Ramp-Up advisors use postsecondary plan- 
ning tools, and that parents are informed about students’ progress on actualizing their career plans. The 
consortium also may want to consider the importance of allowing schools adequate time to implement the 
program as well as the amount of exposure to the program that students receive. A single year of exposure 
may be insufficient to expect a substantial change in college readiness among students. For example, the 
exposure of grade 12 students, who begin the college application process in the fall, may have been too 
little and too late to produce impacts. And Ramp-Up leadership teams, coordinators, and advisors may 
improve implementation of Ramp-Up as they become more familiar with the curriculum. Thus, students in 
grades 10-11 (and earlier grades) will benefit most from several years of Ramp-Up. 
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Implications for school leaders and staff currently implementing Ramp-Up 


Teachers serving as Ramp-Up advisors for the first time may need additional preparation time to effectively 
facilitate the advisory sessions and workshops. Parents and guardians, whose role is critical in motivat- 
ing students to attend college and helping students attain the credentials, abilities, and experiences that 
colleges are seeking, may benefit from more open communication with Ramp-Up advisors and guidance 
counselors about students’ readiness for college. School leaders and staff in Ramp-Up schools may need 
to adjust expectations about the ability of Ramp-Up to improve student outcomes during the first year of 
implementation: as noted above, students will likely benefit from multiple years of exposure to the program. 


Implications for school and district leaders considering adopting Ramp-Up 


The implications for school leaders and staff currently implementing Ramp-Up also apply to school and 
district leaders considering adopting Ramp-Up: schools must commit to implementing all components with 
fidelity for Ramp-Up to have its intended impacts. It is unlikely that Ramp-Up will produce immediate 
impacts on students’ personal readiness for college or the likelihood of key enrollment actions during the 
first year of implementation. Thus, school and district leaders may need to commit to at least two years of 
implementation before drawing conclusions about the program’s immediate and intermediate impacts— 
and possibly longer for long-term impacts. 


Implications for policymakers who want to improve students’ college readiness 


One of the most popular resources for recommendations for improving students’ readiness is the What 
Works Clearinghouse’s practice guide, Helping Students Navigate the Path to College: What High Schools Can 
Do (Tierney, Bailey, Constantine, Finkelstein, & Hurd, 2009). The recommendations in the practice guide 
are: 

e Offer courses and curricula that prepare students for college-level work. 

e Use assessment measures to track students’ level of preparation for college. 

e Surround students with peers who also support college-going aspirations. 

e Assist students in completing the steps for college entry. 

e Increase families’ financial awareness and help students apply for financial aid. 


Although the College Readiness Consortium developed Ramp-Up prior to the publication of the practice 
guide, the five dimensions of readiness that serve as Ramp-Up’s foundation align closely with the practice 
guide recommendations (see box 1). A single program, Ramp-Up involves activities designed to bolster 
students’ academic, admissions, career, financial, and personal—social readiness. However, as the findings of 
this study indicate, Ramp-Up is unlikely to produce immediate impacts within the first year of implementa- 
tion, especially among students further along in their academic careers. Therefore, policymakers may need 
to give schools and districts the ability to make a multiyear commitment to implementing programs such as 
Ramp-Up. An evaluation of Ramp-Up that includes a second year of implementation would provide stron- 
ger evidence of whether Ramp-Up has an impact on students’ enrollment and success in college. 


Limitations of the study 


Four main limitations should be kept in mind when interpreting the study findings. 


First, the study samples were problematic in two ways: the number of students per school was smaller than 
expected, leading to reduced ability to detect smaller effects with statistical confidence, and for some out- 
comes the student-level response rates differed between Ramp-Up schools and comparison schools. This 
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impact study was originally designed to detect effects as small as 5 percentage points for binary outcomes 
and 0.17 standard deviation unit for continuous outcomes. However, given the recruitment of schools with 
fewer students and lower than expected response rates, the study could detect only larger effects—15 per 
centage points (for binary outcomes) and 0.19 standard deviation unit for continuous outcomes—with 
adequate power. 


Response rates for fall and spring staff surveys were 50—70 percent, which some methodologists would con- 
sider barely acceptable (Mangione & Van Ness, 2009). Typically, such rates would indicate that the find- 
ings based on these surveys (in this case, the findings on implementation fidelity) should be viewed with 
caution. However, the findings from completed staff surveys were consistent with those in the previous 
Ramp-Up study on implementation (Lindsay et al., 2016). 


The differential response rates on the ACT Engage goal striving scale and commitment to college scale 
between students in Ramp-Up schools (54 percent) and students in comparison schools (62 percent) also 
suggest that the impacts on those two measures should be interpreted with caution. 


Second, the study used students’ self-reports to measure whether they completed the FAFSA and submit- 
ted at least one college application because data could not be obtained from government agency data- 
bases, such as Minnesota’s state longitudinal data system or the National Student Clearinghouse, because 
of privacy concerns. Participating schools did not adequately track students’ transcript requests for college 
applications. Thus, these outcomes may have been influenced by student biases. The extent of this bias for 
students in Ramp-Up schools and comparison schools is unknown. 


Third, as noted in the Ramp-Up implementation study (Lindsay et al., 2016), school leaders may need more 
than a single year to secure staff buy-in to the program. And staff may need time to become more familiar 
with the curriculum and its delivery so they can carry out their advisor role with fidelity. Students may 
need multiple years of exposure to Ramp-Up’s curriculum to show impacts on immediate, intermediate, 
and long-term outcomes. This would be especially true for two of the key outcomes: FAFSA completion 
and submission of at least one college application. Future studies could benefit from examining impacts of 
Ramp-Up following multiple years of implementation and student exposure to the curriculum. 


Fourth, findings from this study may not generalize to other types of students or other types of student pop- 
ulations. This study was conducted in high schools in Minnesota and western Wisconsin that volunteered 
to participate in the study. The schools and student populations that participated in the study may differ 
from those in other areas of the United States; thus similar studies of Ramp-Up in other settings could 
yield different findings. 
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Notes 


Given the size of the sample of schools and students, as well as response rates, this study had sufficient 
power to detect impacts of 15 percentage points for this outcome. 
Given the size of the sample of schools and students, as well as response rates, this study had sufficient 
power to detect impacts of 14 percentage points for this outcome. 
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